Skip to content

Conversation

@rdblue
Copy link
Contributor

@rdblue rdblue commented Feb 28, 2022

This adds an implementation of RESTClient that translates REST protocol requests to the Catalog API. This makes it possible to test the RESTCatalog implementation like any other catalog. These tests do not exercise serialization; request and response objects are passed between methods directly.

  • RESTCatalogAdaptor is a simple translation from RESTClient routes to catalog handlers
  • CatalogHandlers accepts request objects, makes the appropriate calls to a catalog, and produces response objects
  • RESTCatalog is a Catalog implementation that translates to calls to RESTClient
  • TestRESTCatalog runs CatalogTests against the RESTCatalog

This also implements load table and update table routes, with new tests in CatalogTests that are passing for REST and JDBC.

To get tests working, this PR includes a few other changes and fixes:

  • Adds equals to CharSequenceSet for assertions
  • Updates TableMetadataParser to handle refs
  • Supports -1 to signal "last added" in schema, partition spec, and sort order changes
  • Fixes TableMetadata history timestamps by detecting whether a snapshot was added
  • Adds applyTo(TableMetadata.Builder) to MetadataUpdate to apply changes from the REST protocol on the server side
  • Adds requirements classes and validations that are used server side to check the loaded metadata

@rdblue rdblue marked this pull request as draft February 28, 2022 17:28
@rdblue rdblue changed the title Add rest catalog adapter Core: Add REST catalog adapter and tests Feb 28, 2022
Assert.assertFalse("Namespace should not exist", catalog.namespaceExists(withDot));
}

@Test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: would it make sense to pull these out into a separate PR so that we get those in sooner? Unless this PR won't take too long to be merged of course

}
}

this.requriedLength = parts.size();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: typo in requriedLength

/**
* Adaptor class to translate REST requests into {@link Catalog} API calls.
*/
public class RESTCatalogAdapter implements RESTClient {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it make sense to have a separate test class for this adapter to make sure routing and other stuff is properly tested?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. I'm actually thinking that this class will probably be moved to tests. What we want to make available is CatalogHandlers. The adapter is just a simple way to go between the RESTClient API and CatalogHandlers.


@Override
public String toString() {
return "CharSequenceSet({" + Streams.stream(iterator()).collect(Collectors.joining(", ")) + "})";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, Collectors.joining seems to also take prefix, suffix argument, instead of adding them manually?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know!

@rdblue rdblue force-pushed the add-rest-catalog-adaptor branch from 0d1d4e7 to 339cc16 Compare March 8, 2022 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants